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Model  Checking 


Pentium  floating  point  bug  (1995):  inspired  Intel  to  model  check  chips 
Now  being  applied  to  software,  as  well 
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Probabilistic  Model  Checking 


Model  Checking  is  purely  boolean;  a  property  is  true  or  false. 
For  some  systems,  we  want  probabilities 
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Many  kinds  exist; 
we  use  Discrete  Time 
Markov  Chains  (DTMC) 
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DTMCs  and  Multi-Agent  Robotic  Systems 


•  Benefits: 

1 .  Performance  vs  physics-based  simulation 

2.  Exact  results.  Given  a  model,  probabilities  are  calculated  exactly 

•  Essential  problems: 

1 .  Modelling  physical  systems  is  difficult 

-  Can’t  just  extract  from  a  design  or  program  code;  must  observe 
system  to  model  it 

Physical  systems  are  continuous.  Probabilistic  Model  Checking  relies 
on  discrete  states 

Given  an  imperfect  model  based  on  finite  observations,  how  does  that 
impact  predictions? 

2.  Robots  interact.  Modelling  an  entire  system  of  multiple  robots  is  hard. 
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Our  Contributions 


1.  Model  robots  individually: 

1 .  observe  and  measure  individual  behavior 

2.  discretize  observations  in  time  and  space,  create  Markov  models 

3.  compose  these  models  into  a  Markov  model  of  the  whole  system 

2.  Use  known  statistical  error  on  the  measurements  made  of  the 
individual  robots  to  produce  estimates  of  error  of  the  outputs  of 
model  checking  the  whole  system. 
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Scenario 


For  our  experiments,  and  as  an  illustration,  we  imagine  a  mine  sweeping 
scenario.  Objective:  find  a  mine/IED  in  a  constrained  space  (i.e.,  a 
drainage  culvert  under  a  road). 


Discrete  Time  Markov  Chains 


DTMCs  have: 

•  A  set  of  states,  each 
representing  a  discrete  point 
in  time 

•  Transitions  between  states, 
with  probabilities  associated.  0.3 
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Modal  DTMCs 


We  contribute  a  variant: 

•  States  can  also  have  “mode 
change”  transitions. 

•  Mode  changes  can  represent 
interaction  between  robots, 

or  the  environment.  MINE 

•  In  our  paper,  we 
show  how  to 
convert  to  a  basic 
DTMC  for  use  with 
existing  model 
checkers. 
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Discrete  Time  and  Space 

We  represent  robot  position  with  states 
representing  different  grid  positions, 
and  different  times. 
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Composing  Modal  DTMCs 


•  Modal  DTMCs  allow  us  to  model  individual  robots,  then  easily 
compose  them  together. 

•  To  create  an  individual  model: 

1 .  Run  the  robots  individually,  with  pre-planned  mode  changes 

2.  Observe  the  robot’s  behavior 

3.  Create  a  Modal  DTMC  with  transition  probabilities  based  on  observation, 
and  mode  changes  as  pre-planned 

•  Then,  collect  the  individual  modal  DTMCs  into  a  whole-system 
modal  DTMC,  and  convert  it  to  a  non-modal  DTMC 

•  Details  of  this  construction,  and  correctness  proof,  are  in  the  paper. 
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Error  Estimation 


•  Probabilistic  Model  Checking  itself  has  no  error;  given  a  model,  it 
finds  exact  probabilities. 


•  However,  modeling  a  robotic  system  will  certainly  not  be  perfect. 


•  Many  kinds  of  error  might  appear  causing  a  model  to  not  reflect 
reality.  We  looked  at  handling  one:  the  statistical  errors  due  to 
observing  the  individual  robots  only  a  finite  number  of  times. 


•  To  examine  this  specific  kind  of  error,  we  assume: 

•  That  the  system  can  be  fully  described  by  a  DTMC 

•  That  we  have  figured  out  the  states  of  that  DTMC 

•  But  the  transition  probabilities  are  observed  over  the  course  of  finite  trials 
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Dirichlet-based  Distribution  of  DTMCs 


To  analyze  error,  we  create  a  random  distribution  of  DTMCs. 

For  each  transition,  we  use  the  counts  of  the  times  that  transition  was 
observed  to  describe  the  Dirichlet  distribution  of  transition  probabilities 
for  that  state,  which  includes  a  variance  which  shrinks  with  more 
observations. 


Model  Checking  a  distribution  of  DTMCs 

We  randomly  generate  a 
large  number  of  DTMCs 
from  the  distribution  we 
created.  We  model  check 
each  DTMC,  which  gives 
us  probabilities  for  our 
properties  of  interest.  This 
gives  us  a  mean  and 


Experiment 


•  Used  the  simulator  V-REP,  with  Kilobot  models  based  on 
observations  of  real  Kilobots 


•  Simulated  Kilobots  individually,  used  observations  to  create  models 
for  various  team  configurations  (using  Modal  DTMCs),  and  predicted 
outcomes,  using  our  Dirichlet  sampling  technique.  Our  metrics: 

•  Probability  base  learns  of  mine  (SUCCESS) 

•  Expected  number  of  bots  that  return  to  base  (RETURNED) 

•  Simulated  those  teams  in  V-REP,  and  compared  those  outcomes  to 
predicted  outcomes. 
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Experiment  Results  -  SUCCESS  metric 


i 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

0 


n 


I 


95-%ile 


5-%ile 


♦  Mean 


♦  Observed 


3-2-1  4-6-1  4-6-2  5-6-2  5-6-7  6-1-7  6-5-7  7-3-5  7-3-6  7-6-1 


=  Software  Engineering  Institute  Carnegie  Mellon  University 


SIMPAR  2014,  Bergamo,  Italy 
David  Kyle,  Oct.  22,  2014 

©2014  Carnegie  Mellon  University 


Experiment  Results  -  RETURNED  metric 
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Questions? 
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